Comparing OpenBRR, QSOS, and OMM Assessment Models

The objective of this study was to investigate the quality and usability of three Free/Libre Open Source Software assessment models: the Open Business Readiness Rating (OpenBRR), the Qualification and Selection of Open Source software (QSOS), and the QualiPSo OpenSource Maturity Model (OMM). The study identified the positive and negative aspects of each of them. The models were used to assess two Free/Libre Open Source Software projects: Firefox and Chrome (Chromium). The study is based on a set of controlled experiments in which the participants performed the assessment using only one model each. The model used and the Free/Libre Open Source Software project assessed were randomly assigned to the participants. The experiment was conducted in a controlled environment with defined tasks to be performed in a given time interval. The results revealed that the three models provided comparable assessments for the two assessed projects. The main conclusion was that all the three models contain some questions and proposed answers that are not clear to the assessors, therefore should be rewritten or explained better. The critical aspects of each model were: Functionality and Quality for OpenBRR; Adoption, Administration/Monitoring, Copyright owners, and Browser for QSOS ; and Quality of the Test Plan, and the Technical Environment for OMM. Participants perceived the quality and usability of the three models of comparable level.